1. Field of the Invention
The present invention relates to an image recognition apparatus and method for recognizing the shape and/or movement of an image based on a captured distance image.
2. Discussion of the Background
In recognizing a shape or a movement of the lips of a human or judging a direction, expression, or the like of the face of a human using image processing technology, first, the lips and their vicinity or a face portion of a human is imaged by using an imaging apparatus such as a CCD camera. Then, preprocessing is performed by cutting out only an object to be recognized such as only a lip portion or only a face portion from a resulting image by removing unnecessary portions such as the background. A shape or a movement is recognized by using an image obtained by the preprocessing.
In conventional preprocessing, an object is cut out by using, as a key, some difference between the object and the other portions. There are several methods that use, as a key, a variation in hue, a difference image, a marker or the like, or a chromakey.
However, an image that has been cut out by any of the above methods includes only two-dimensional information. Even if there are imaging apparatuses that acquire a three-dimensional shape, they are not suitable for real-time recognition such as recognition of a movement. There is another problem that the three-dimensional imaging apparatus is very expensive and cannot be used easily. For this reason, in the conventional image processing, a shape or movement of the face or lips of a human is recognized by using only two-dimensional information.
However, since a shape or a movement that is three-dimensional in nature is handled as two-dimensional information, it cannot be recognized accurately because necessary information is omitted and only a simple shape or movement can be recognized in spite of various measures taken.
As described above, conventionally, it is very difficult to accurately cut out only a desired object portion from an image obtained by a camera, and this is one factor which lowers the recognition rate of image recognition.
Since an image is captured as two-dimensional information by using a camera or the like, because of various limitations here is no other way than recognizing a three-dimensional shape or movement based on only two-dimensional information. This causes a problem that only a simple shape or movement can be recognized.
The present invention has been made in view of the above problems in the art, and an object of the invention is therefore to provide an image recognition apparatus and method which can recognize a shape or a movement of the face or lips of a human at high speed with high accuracy.
The above and other objects are achieved according to the present invention by providing a novel image recognition apparatus including an image capturing device which captures a distance image of an object; a mouth portion extracting mechanism configured to extract a mouth portion from the distance image captured by the image capturing device; and an image recognizing mechanism configured to recognize a lip shape based on a mouth portion distance image extracted by the mouth portion extracting mechanism.
According to a second aspect of the invention, there is provided an image recognition apparatus including an image capturing device which captures a distance image stream of an object; a mouth portion extracting mechanism configured to extract a mouth portion from the distance image stream captured by the image capturing device; and an image recognizing mechanism configured to recognize a lip movement based on a mouth portion distance image stream extracted by the mouth portion extracting mechanism.
According to a third aspect of the invention, there is provided an image recognition apparatus including an image capturing device which captures a distance image of an object; a face portion extracting mechanism configured to extract a face portion from the distance image captured by the image capturing device; and an image recognizing mechanism configured to recognize a face shape based on a face portion distance image extracted by the face portion extracting mechanism.
According to a fourth aspect of the invention, there is provided an image recognition apparatus including an image capturing device which captures a distance image stream of an object; a face portion extracting mechanism configured to extract a face portion from the distance image stream captured by the image capturing device; and an image stream captured distance image; and recognizing a lip shape based on an extracted mouth portion distance image.
According to a sixth aspect of the invention, there is provided an image recognition method including the steps of capturing a distance image stream of an object; extracting a mouth portion from the captured distance image stream; and recognizing a lip movement based on an extracted mouth portion distance image stream.
According to a seventh aspect of the invention, there is provided a computer-readable memory containing computers readable instructions, including a first instruction which causes a computer to capture a distance image of an object; a second instruction which causes the computer to extract a mouth portion from the captured distance image; and a third instruction which causes the computer to recognize a lip shape based on an extracted mouth portion distance image.
According to an eighth aspect of the invention, there is provided a computer-readable memory containing computer-readable instructions, including a first instruction which causes a computer to capture a distance image stream of an object; a second instruction which causes the computer to extract a mouth from the captured distance image stream; and a third instruction which causes the computer to recognize a lip movement based on an extracted mouth portion distance image stream.